Shaping the auditory peripersonal space with motor planning in immersive virtual reality

Immersive audio technologies require personalized binaural synthesis through headphones to provide perceptually plausible virtual and augmented reality (VR/AR) simulations. We introduce and apply for the first time in VR contexts the quantitative measure called premotor reaction time (pmRT) for characterizing sonic interactions between humans and the technology through motor planning. In the proposed basic virtual acoustic scenario, listeners are asked to react to a virtual sound approaching from different directions and stopping at different distances within their peripersonal space (PPS). PPS is highly sensitive to embodied and environmentally situated interactions, anticipating the motor system activation for a prompt preparation for action. Since immersive VR applications benefit from spatial interactions, modeling the PPS around the listeners is crucial to reveal individual behaviors and performances. Our methodology centered around the pmRT is able to provide a compact description and approximation of the spatiotemporal PPS processing and boundaries around the head by replicating several well-known neurophysiological phenomena related to PPS, such as auditory asymmetry, front/back calibration and confusion, and ellipsoidal action fields

Classifying non-individual head-related transfer functions with a computational auditory model: calibration and metrics

This study explores the use of a multi-feature Bayesian auditory sound localisation model to classify non-individual head-related transfer functions (HRTFs). Based on predicted sound localisation performance, these are grouped into ‘good’ and ‘bad’, and the ‘best’/‘worst’ is selected from each category. Firstly, we present a greedy algorithm for automated individual calibration of the model based on the individual sound localisation data. We then discuss data analysis of predicted directional localisation errors and present an algorithm for categorising the HRTFs based on the localisation error distributions within a limited range of directions in front of the listener. Finally, we discuss the validity of the classification algorithm when using averaged instead of individual model parameters. This analysis of auditory modelling results aims to provide a perceptual foundation for automated HRTF personalisation techniques for an improved experience of binaural spatial audio technologies

Localization in elevation with non-individual head-related transfer functions: Comparing predictions of two auditory models

This paper explores the limits of human localization of sound sources when listening with non-individual Head-Related Transfer Functions (HRTFs), by simulating performances of a localization task in the mid-sagittal plane. Computational simulations are performed with the CIPIC HRTF database using two different auditory models which mimic human hearing processing from a functional point of view. Our methodology investigates the opportunity of using virtual experiments instead of time- and resource- demanding psychoacoustic tests, which could also lead to potentially unreliable results. Four different perceptual metrics were implemented in order to identify relevant differences between auditory models in a selection problem of best-available non-individual HRTFs. Results report a high correlation between the two models denoting an overall similar trend, however, we discuss discrepancies in the predictions which should be carefully considered for the applicability of our methodology to the HRTF selection problem

Shaping the auditory peripersonal space with motor planning in immersive virtual reality

Immersive audio technologies require personalized binaural synthesis through headphones to provide perceptually plausible virtual and augmented reality (VR/AR) simulations. We introduce and apply for the first time in VR contexts the quantitative measure called premotor reaction time (pmRT) for characterizing sonic interactions between humans and the technology through motor planning. In the proposed basic virtual acoustic scenario, listeners are asked to react to a virtual sound approaching from different directions and stopping at different distances within their peripersonal space (PPS). PPS is highly sensitive to embodied and environmentally situated interactions, anticipating the motor system activation for a prompt preparation for action. Since immersive VR applications benefit from spatial interactions, modeling the PPS around the listeners is crucial to reveal individual behaviors and performances. Our methodology centered around the pmRT is able to provide a compact description and approximation of the spatiotemporal PPS processing and boundaries around the head by replicating several well-known neurophysiological phenomena related to PPS, such as auditory asymmetry, front/back calibration and confusion, and ellipsoidal action fields

Action planning and affective states within the auditory peripersonal space in normal hearing and cochlear-implanted listeners

Fast reaction to approaching stimuli is vital for survival. When sounds enter the auditory peripersonal space (PPS), sounds perceived as being nearer elicit higher motor cortex activation. There is a close relationship between motor preparation and the perceptual components of sounds, particularly of highly arousing sounds. Here we compared the ability to recognize, evaluate, and react to affective stimuli entering the PPS between 20 normal-hearing (NH, 7 women) and 10 cochlear-implanted (CI, 3 women) subjects. The subjects were asked to quickly flex their arm in reaction to positive (P), negative (N), and neutral (Nu) affective sounds ending virtually at five distances from their body. Pre-motor reaction time (pm-RT) was detected via electromyography from the postural muscles to measure action anticipation at the sound-stopping distance; the sounds were also evaluated for their perceived level of valence and arousal. While both groups were able to localize sound distance, only the NH group modulated their pm-RT based on the perceived sound distance. Furthermore, when the sound carried no affective components, the pm-RT to the Nu sounds was shorter compared to the P and the N sounds for both groups. Only the NH group perceived the closer sounds as more arousing than the distant sounds, whereas both groups perceived sound valence similarly. Our findings underline the role of emotional states in action preparation and describe the perceptual components essential for prompt reaction to sounds approaching the peripersonal space

Initial evaluation of an auditory-model-aided selection procedure for non-individual HRTFs

Binaural spatial audio reproduction systems use measured or simulated head-related transfer functions (HRTFs), which encode the effects of the outer ear and body on the incoming sound to recreate a realistic spatial auditory field around the listener. The sound localisation cues embedded in the HRTF are highly personal. Establishing perceptual similarity between different HRTFs in a reliable manner is challenging due to a combination of acoustic and non-acoustic aspects affecting our spatial auditory perception. To account for these factors, we propose an automated procedure to select the ‘best’ non-individual HRTF dataset from a pool of measured ones. For a group of human participants with their own acoustically measured HRTFs, a multi-feature Bayesian auditory sound localisation model is used to predict individual localisation performance with the other HRTFs from within the group. Then, the model selection of the ‘best’ and the ‘worst’ non-individual HRTFs is evaluated via an actual localisation test and a subjective audio quality assessment in comparison with individual HRTFs. A successful model-aided objective selection of the ‘best’ non-individual HRTF may provide relevant insights for effective and handy binaural spatial audio solutions in virtual/augmented reality (VR/AR) applications

Evaluation of spatial tasks in virtual acoustic environments by means of modeling individual localization performances

Virtual acoustic environments (VAEs) are an excellent tool in hearing research, especially in the context of investigating spatial-hearing abilities. On the one hand, the development of VAEs requires a solid evaluation, which can be simplified by applying auditory models. On the other hand, VAE research provides data, which can support the further improvement of auditory models. Here, we describe how Bayesian inference can predict listeners' behavior when estimating the spatial direction of a static sound source presented in a VAE experiment. We show which components of the behavioral process are reflected in the model structure. Importantly, we highlight which acoustic cues are important to obtain accurate model predictions of listeners' localization performance in VAE. Moreover, we describe the influence of spatial priors and sensorimotor noise on response behavior. To account for inter-individual differences, we further demonstrate the necessity of individual calibration of sensory noise parameters in addition to the individual acoustic properties captured in head-related transfer functions

A Bayesian model for human directional localization of broadband static sound sources

Humans estimate sound-source directions by combining prior beliefs with sensory evidence. Prior beliefs represent statistical knowledge about the environment, and the sensory evidence consists of auditory features such as interaural disparities and monaural spectral shapes. Models of directional sound localization often impose constraints on the contribution of these features to either the horizontal or vertical dimension. Instead, we propose a Bayesian model that flexibly incorporates each feature according to its spatial precision and integrates prior beliefs in the inference process. The model estimates the direction of a single, broadband, stationary sound source presented to a static human listener in an anechoic environment. We simplified interaural features to be broadband and compared two model variants, each considering a different type of monaural spectral features: magnitude profiles and gradient profiles. Both model variants were fitted to the baseline performance of five listeners and evaluated on the effects of localizing with non-individual head-related transfer functions (HRTFs) and sounds with rippled spectrum. We found that the variant equipped with spectral gradient profiles outperformed other localization models. The proposed model appears particularly useful for the evaluation of HRTFs and may serve as a basis for future extensions towards modeling dynamic listening conditions

Predicting Directional Sound-Localization of Human Listeners in both Horizontal and Vertical Dimensions

Measuring and understanding spatial hearing is a fundamental step to create effective virtual auditory displays (VADs). The evaluation of such auralization systems often requires psychoacoustic experiments. This process can be time consuming and error prone, resulting in a bottleneck for the evaluation complexity. In this work we evaluated a probabilistic auditory model for sound localization intended as a tool to assess VAD's abilities to provide static sound-localization cues to listeners. The outcome of the model, compared with actual results of psychoacoustic experiments, shows the advantages and limitations of this systematic evaluation